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This paper presents the role of Independent Assessment in the International 
Space Station (ISS) Program. Independent Assessment is responsible for 
identifying and specifying technical and programmatic risks that may impact 
development, launch, and on-orbit assembly and operations of the ISS. The 
various phases of the assessment process are identified and explained. 

This paper also outlines current and future participation by Independent 
Assessment in Human Exploration and Development of Space projects 
including the X-38 Space Plane , Mars mission scenarios, and applications of 
Nanotechnology. 

This paper describes how Independent Assessment helps the shuttle, ISS, 
and other programs to safely achieve mission goals now and into the next 
century. 

INTRODUCTION AND BACKGROUND 

This paper presents the role of Independent Assessment (IA) in the International Space 
Station Program (ISSP). This overview also identifies, describes, and explains future plans for 
IA involvement in the Human Exploration and Development of Space (HEDS) initiatives. 

The International Space Station (ISS) is a multidisciplinary laboratory, testbed, and 
observatory that will provide unprecedented opportunities in technology and scientific 
experimentation. The ISS requires that design, hardware, logistics, management, and operations 
all be integrated. For a mission of this grand scale, some of the ISS assembly elements will be 
the heaviest cargo items ever sent into space. The ISS assembly requires an unprecedented 
number of Extra Vehicular Activities (EVA), rendezvous and docking, and robotic remote 
manipulator system activities. There will be many new challenges inherent in assembling the 
most complex space structure ever created. 

Beginning in 1994, 1 A activities have been focused on assuring the safety and integrity 
of the ISS while attaining mission objectives. IA is responsible for identifying and specifying 
technical and programmatic risks that may impact successful ISS development, launch, and on- 
orbit assembly and operations. The I A Director reports directly to the Office of Safety and 
Mission Assurance (S&MA) at NASA Headquarters. 

IA assessments provide inputs to the ISSP decision making process by identifying key 
risks in a timely fashion. IA contributes unbiased evaluations based on analysis and testing. 
They focus on technical issues rather than Program budget or schedule concerns. 

Customers include ISSP managers, the NASA Administrator, and other NASA senior managers. 
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Our Vision and Mission 

The Independent Assessment organization is recognized as a proactive, technically proficient 
team that provides meaningful, timely, candid assessments that are relevant to the success of the 
ISSP. 

RISK MANAGEMENT 

A primary objective of Independent Assessment is to assist the ISSP in risk management 
and mitigation. Risk management is an organized, systematic decision-making process that 
effectively identifies risks, assesses or analyzes risks, and effectively reduces or eliminates risks 
to achieve program goals. 

A risk is an undesirable situation or circumstance that has both a probability of occurring 
and a potential consequence to program success. 

Table 1 (see appendix) defines risk factor values for likelihood and consequence. 
Likelihood is the chance a situation or circumstance will happen. Consequence is the magnitude 
of the impact if the condition occurs. Figure 1 is a diagram of the ISSP risk matrix display. 

Figure 1 

ISSP Risk Matrix 
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This 5-by-5 matrix has three risk zones labeled as high, medium, and low. These zones are 
defined as follows: 

High Risk - Likely to cause significant and/or serious disruption of schedule, increase in 
cost, or degradation of performance even with special contractor emphasis and close monitoring. 

Medium Risk - Potentially can cause some disruption of schedule, increase in cost, or 
degradation of performance; however, special contractor emphasis and close monitoring will 
probably be sufficient to overcome difficulties. 

Low Risk - Little or no potential for disruption of schedule, increase in cost, or 
degradation of performance; normal contractor effort and normal monitoring will probably be 
sufficient to overcome difficulties. 

The risk level is determined from the likelihood and consequence values. Table 2 (see 
appendix) defines IA categories of identified risks. 
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Figure 2 

Risk Management Flow Chart 



The flow of 1SS Program risk management, as shown in Figure 2, is comprised of 5 

steps: 

1 ) Risk Identification - What key technical area or process is at risk? 

2) Risk Analysis - Determine the root cause of the risk. Quantify your risks by deter- 
mining the likelihood of an event and the potential consequence to the ISS. 

3) Risk Abatement - What can you do about a risk? Identify possible solutions. Next, 
develop a mitigation/contingency plan or accept the risk. 

4) Risk Communication - Provide status of the risks on a regular basis (done monthly 
for ISS) 

5) Monitor progress of the risk mitigation 

FLIGHT LEADS AND SYSTEM SPECIALISTS 

A Flight Lead in the IA organization is responsible for all technical aspects associated 
with a particular flight/mission in the Assembly Sequence. The purpose of the missions vary 
from delivering ISS hardware elements to orbit, assembly support, or resupply by an 
International Partner/Participant. A Flight Lead is the IA expert and Point-of-Contact for 
specific questions about a particular flight. The Flight Leads act in a consulting role to the 
Independent Assessment Office. They identify and track issues, problems, and risks associated 
with their flight. Currently, there is a Flight Lead for ISS 2A.1 through 7A. 

System Specialists from Operations, Test and Verification (T&V), Thermal and 
Structures, and Environmental/EVA teams provide information and data to each Flight Lead. 
They are the expert and Point-of-Contact for specific questions for a particular subsystem or 
process. 

Flight Leads and System Specialists use the Risk Assessment Matrix to define the level 
of risks associated with a flight. This information is maintained in the IA Risk Database that is 
located on the IA Web Home Page. Each Flight Lead also maintains a weekly and monthly 
health summary for their mission. Inputs are provided from each of the Operations and System 
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Specialist disciplines and integrated by the Flight Lead. An example from Flight 2A.I is 
displayed in Figure 3 (see Appendix). 

ELEMENTS IN THE TECHNICAL ASSESSMENT PROCESS 

The phases of the assessment process include: 

a) Planning the Assessment 

b) Performing the Assessment 

c) Documenting the Assessment 

d) Tracking Implementation of Recommendations. 

The Assessment Proposal 

All technical areas of the Program are reviewed and evaluated for potential assessment. 
The assessment proposal defines the purpose, approach, and scope of an assessment. It identifies 
what parameter, question, or measure will be evaluated to meet the objective of the assessment. 

Performing an Assessment 

Performing an assessment involves gathering data and information on the subject. This 
research may include reviewing documentation, attending various meetings, panels, or forums, 
circulating questionnaires, and conducting interviews with a wide variety of personnel. IA 
personnel may perform their own test or analyses. These facts and calculations are used in 
determining if defined criteria are met. They provide the basis for developing IA findings and 
recommendations. These findings lead to conclusions about the adequacy or risk of the subject 
that evolve into recommendations. 

Presentation of IA Results, Findings and Recommendations 

Findings and recommended actions based on results of an assessment are coordinated 
with the Program through an assigned Point-of-Contact. A response to these findings and 
recommendations is provided by Program personnel from the area associated with the 
assessment topic. The assessment author then tracks implementation of the recommendations. In 
some cases the recommendations and proposed resolution may also be presented to a Program 
board or panel overseeing the subject of the assessment. 

Certificate of Flight Readiness Input 

IA participates in the ISSP Certificate of Flight Readiness (CoFR) Review. CoFR is the 
culmination of a continuous process of Program review and evaluation of the ISS requirements, 
design implementation, mission planning, and processes. CoFR is conducted to determine if the 
hardware, software, procedures, training, and technical support is ready for flight. It certifies the 
on-orbit stage and operations readiness for each flight. The IA participation in CoFR is iterative 
including assessments of specific topics to participation in ISSP forums with real time decisions. 
It includes flight-related assessments, tracking Program risks, and participation in various 
Program meetings. Risks identified as a result of IA activity are worked both formally and 
informally with the Program. Flight Leads track risks for each flight to support the CoFR 
decision. Status of flight risks are updated by interim reports, presentations, and status briefings 
at specific times to support earlier CoFR reviews. Signature by the IA Manager who represents 
the IA Director to the ISS Program signifies concurrence that IA activity has revealed nothing 
that will preclude safe and successful launch and on-orbit operations. 
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RECENT CONTRIBUTIONS 

Topics of recent assessments include: 

a) Lessons Learned from ISS Phase I 

b) Space Vision System 

c) Electrical Power System ORUs and System Performance Concerns 

d) ISS Truss Segment P6 Thermal Dissipation 

e) Structural Analysis of the X-38 Spaceplane Parafoil 

f) ISS and Joint ISS/Shuttle Flight Rules 

g) Shuttle Readiness to Support ISS Assembly Flights 2A through 7A 

h) Problem Reporting and Corrective Action System for the ISSP 

i) ISS Government Furnished Equipment 

j) Reliability and Maintainability of ISS Critical Items 

INDEPENDENT VERIFICATION AND VALIDATION 

The Software Independent Verification and Validation (IV&V) group within the IAO is 
responsible for assessment of mission critical software to ensure adequate performance and 
reliability. This includes a variety of activities from requirements development phase through 
formal qualification testing and follow-on integration testing. 

IV&V is a process whereby the products of the software development life cycle phases 
are independently reviewed, verified, and validated by an organization independent of the 
developers and acquirers of the software. These activities are accomplished on the U.S. segments 
for designated catastrophic, critical, or high risk software systems. IV&V employs a Criticality 
Analysis and Risk Assessment (CARA) to identify which components of the function in question 
require IV&V and to what extent. Criticality is a measure of the potential impact to the program 
of an error in the specification or implementation of a function. Risk is a measure of the 
likelihood of occurrence of such an error. The CARA then serves as a quantitative aid to the 
effective allocation of IV&V resources. 

The ISS will be developed and assembled in stages. Each stage is a complete, self- 
contained operating vehicle. For this reason, the stage aspects of development are an important 
concept in the ISS IV&V plan. IV&V activities are repeated for each stage or group of stages. 

Contributions from NASA Centers 

Analysts are resident at the Johnson Space Center, Kennedy Space Center, and Marshall 
Space Flight Center to support assessments at these sites. Lewis Research Center personnel 
perform parallel activities for Boeing in Huntington Beach and Canoga Park. Other IA members 
are located at NASA Headquarters in Washington, D.C. and the IV&V facility at Fairmont, W.V. 

Safety Review Panel and ISO 9000 

A representative from the Independent Assessment Team is a voting member of the ISS 
Safety Review Panel (SRP). This group evaluates potential safety hazards associated with the 
assembly or operation of the ISS. The IA representative discusses potential hazards with the 
Flight Leads and addresses their issues and concerns to the panel. 

Since NASA/JSC is ISO 9000 certified, IA follows the process in place for 
documentation and performance of its activities. 

Integration of OnOrbit Assembly and Operations 

Effective October, 1998, the Independent Assessment Office became responsible for 
independent assessments of the Space Shuttle Program. One of the greatest challenges is to 
overcome the non-integrated problems between shuttle and station especially with on-orbit 
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assembly and operations. Towards this goal, I A is performing an assessment that examines the 
processes for S&MA early involvement on late Shuttle and Space Station changes. This 
evaluation was an outcome of the STS-96/ISS2A. 1 flight. On that mission, the biggest challenge 
was keeping abreast of the late manifest revisions and an unsettled timeline. 

IA was instrumental in devising a test plan for OSVS operations from the ground while 
the crew slept. This approach minimized the amount of crew time and involvement in the DTO 
on a flight that was already heavily ‘'booked’' in terms of crew timeline and workload. 

Threats and concerns that I A are tracking for STS-101/ISS 2A.2 include: 

a) Crew concern about number, priority, procedures, and training for EVA and IVA 
installation tasks and hardware 

b) On-orbit stowage shortfall 

c) Definition of Orbiter Space Vision System test requirements 

d) Noncompliance of Service Module acoustics including the Treadmill with Vibration 
Isolation and Stabilization (TVIS) 

e) Service Module Outfitting 

Enough time for installation tasks and logistics stowage? 

Enough crew participation in developing procedures and training actual 

tasks in a flight rated vehicle 

Lack of high fidelity mockup 

Installation tasks : list of tasks complete? Timely? 

Tools - integration and management of SS, ISS, and Russian tools for 
installation tasks 

f) Stowage and Installation Integration - highly dependent on each other 

g) EVA - Lack of integration between various US and Russian groups for training 
plans, hardware testing, procedures, and timelining further hindered by a lack of 
perspective and authority to implement action 

h) EVA - tasks are performed in priority order but not necessarily the most efficient 
method and may increase overall risk 

i) EVA — a very compressed schedule for EVA task development, testing, and training 
with much of it performed in Russia 

j) the risk of having to unman the station after crew arrival for the first system failure, 
or a failure to get critical systems activated within the consumables reserves, or 
depleting your contingency reserves before you have even started the increment 

In general, many late changes to 2A.1 and 2A.2 manifests and tasks, plus late receipt of 
hardware and certification data raise concerns for adequate review, training and overall mission 
planning requiring oversight by NASA and Boeing 

A summary of the major concerns are listed below for each of the follow-on flights: 

3 A - determine operational and attitude constraints for Launch-to- Activation operations 
4A - Full impact of thermal environment on ORUs not known 

5 A - Software issues, berthing operations concerns, and numerous hardware problems must be 
surmounted. 

5A. I - Free drift constraints are undefined. 

6A - Incomplete definition of tasks to be performed on ISS 5A.1 and 6A 
7A - GFE : various problems with the Battery Charge Assembly, Power Supply Assembly, and 
Collapsible Water Reservior. 

Interim Control Module - Will it fly? RF antenna and Star Tracker thermal failures 
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Human Exploration and Development of Space and the Future 

Early in the next Century, NASA plans to return to the Moon and explore Mars and 
beyond. Current and future plans for IA’s involvement in HEDS include the X-38 Crew Return 
Vehicle, X-34 Reusable Launch Vehicle, various Lunar and Mars mission scenarios, and 
applications of Nanotechnology. IA will support the HEDS requirements for developing new 
technologies while mitigating risks. HEDS provides a valuable springboard to design, develop, 
test, and evaluate the technologies that will make these missions possible. The next generation of 
planetary explorers has begun to fly aboard the Shuttle today. 

Unique Contributions and Value to Customer 

IA provides the following contributions to their customers through its assessments: 

a) Provides management with an objective review and unbiased perspective of Program 
activities 

b) Provides a means for checks and balances of decisions 

c) Provides a technical expertise to assist with special problems 

d) Ensures the Program is operating safely and competently by challenging designs, 
requirements, processes, and products 

e) Insulated from most budget, schedule, or political influences 

Added value to the Program from these contributions comes in the form of improved 
processes, training, maintenance, mockups or simulations, data or information flow, or flight 
systems. Thus, greater confidence is established in mission safety and integrity . 

World Wide Web Site 

For more information on IA, see this web site: wwwsrqa.jsc.nasa.gov/ia/HEDS.htm. 

Note this site is not accessible to the public. Access may be granted on an individual need basis. 

SUMMARY AND CONCLUSION 

IA establishes confidence in the safety and mission integrity of NASA programs and 
projects. It identifies problems or issues that might otherwise go unnoticed until they result in a 
mishap or failure. These services help prepare the Program to safely operate the shuttle and 
develop, assemble, and operate the ISS to achieve mission goals and success now and into the 
next century. 
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Table 1 


Likelihood vs. Consequence 


Likelihood 


Consequence 


0 - Not Credible 0 - None 


1 - Extremely Unlikely 1 - 1 st Aid Injury; Loss of ORU 

(small number, recoverable); may 
require alternative procedure 

2 - Unlikely 2 - Injury (can function in most 

tasks); some short term loss of 
• mission capability; requires extra 

EVA, IVA, etc. 

3 - 50/50 Chance 3 - Injury (treatable on-orbit but not 

fully capable); <50% loss 
mission capability (recoverable); 
cannot complete assembly tasks 
(deferred completion) 

4 - Probable 4 - Severe Injury / minor permanent 

disability (requires evacuation); 
Loss of element / Loss of > 50% 
Mission capability (recoverable); 
Inability to assemble a given 
element (return to earth for 
repair) 

5 - Will Occur 5 - Death/Permanent Severe 

Disability; Loss of ISS/Capability 
(non-recoverable); Inability to 
continue assembly 
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Table 2 


IAO Categories of Identified Risks 

Unacceptable Risk . A safety / mission success risk designated by the IA Manager to be an 
exception to the ISS Certification of Flight Readiness (CoFR). The ISS Program mitigation plan 
will not likely resolve the risk by the launch date. 

Potentially Unacceptable Risk . A safety / mission success risk that may be elevated to an 
unacceptable status in the future. This risk will be classified as either an issue or a concern. An 
issue is a risk that IA believes is either not being addressed by the Program or has a defined 
mitigation plan that is inadequate or is not being implemented adequately. A concern is a risk 
that the Program acknowledges and IA believes has an adequate mitigation plan which is being 
implemented (if enough time has elapsed since the risk was identified). 

Watch Item . A safety / mission success risk which IA does not expect to raise to an 
unacceptable status because the Program has a credible mitigation plan in place which is 
reasonably on schedule and results are good. This risk is serious enough to warrant tracking to 
completion. 

Acceptable Risk . A safety / mission success risk which IA does not expect to raise to an 
unacceptable status because either it has low likelihood and consequence scores or it has a high 
score but IA agrees with the Program's risk acceptance rationale. 

Closed Risk . A safety / mission success risk which IA believes has been resolved. Only the risks 
closed since the previous review are identified within this presentation package. 
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Figure 3 

Flight 2A.1 Status 
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Electrical Power Systems 


• No Known Issues 

Engineering 

Y 

• Removal & reinstallation of Common Berthing Mechanism (CBM) controllers 
in PMA-2 is a concern based on past experience and performance 

• Sequential Shunt Unit in rework 

International Partner/Participant 


• 98% of drawings of Russian H/W delivered for Stowage planning 

Launch Processing Integration 

• 1 week slip in launch date to May 20 due to STS-93/AXAF but no technical 

impact 

Operations 

Y* 

• Late manifest revisions and unsettled manifest 

• Procedure for controlling the FGB (without Service Module) during docking 
operations with an APAS lacks detail (software patch in work) 

• Stowage Layout presented at Jan. 26 SDOM showed positive (10-15 
Middeck Locker Equivalent) margin 

Payloads 

Not App 


Safety 

Y 

• No Known Issues 

Shuttle Integration 

• No Known Issues 

Software 

• Orbiter Space Vision System (version 4.2) : late delivery may adversely 
affect quality of training 

Systems 

Y* 

• Questionable reliability of CBM controllers after reinstallation 

Test and Verification 


• No Known Issues (100% of T&V requirements submitted to NASA) 

Thermal/Structures 

• No Known Issues 

Mission Integration 

KQ| 

• Late manifest revisions and unsettled manifest continue but no technical 
impact 


Acceptable - Credible mitigation plans 
are in place, or low likelihood and 
consequences scores, or agreement to 
accept risks. 


ra 

— Potentially Unacceptable - 
Inadequate/incomplete resolution. 


^ Improving 


4' Worsening 


E3 

™ Unacceptable - Not likely to be 
resolved, an exception to the ISS 
CoFR. 
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Acronyms 


CARA 

Criticality Analysis and Risk Assessment 

CoFR 

Certificate of Flight Readiness 

EVA 

Extra Vehicular Activity 

HEDS 

Human Exploration and Development of Space 

IA 

Independent Assessment 

ISO 

International Standards Organization 

ISSP 

International Space Station Program 

IV&V 

Independent Verification and Validation 

NASA 

National Aeronautics and Space Administration 

S&MA 

Safety and Mission Assurance 

SRP 

Safety Review Panel 

SVS 

Space Vision System 

T & V 

Test and Verification 

Y2K 

Year 2000 
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